Photosensor array gesture detection

ABSTRACT

Photosensor array gesture detection techniques are described. In one or more embodiments, a computing device includes an array of photosensors. The photosensor array can be configured in various ways to measure changes in the amount of light that occur based upon a user&#39;s hand position above the photosensor array. In at least some embodiments, capacitance associated with the photosensors is charged and data regarding discharge rates for the sensors is collected that is indicative of the amount of incident light. Sequential changes in the amount of light that is measured across the array of photosensors can be used to determine positioning and/or movement of the user&#39;s hand in three dimensions (e.g., track position/motion in three-dimensional (3D) space relative to the computing device.) Accordingly, various gestures can be defined in terms of input obtained via the photosensor array and recognized to trigger corresponding operations by the computing device.

BACKGROUND

One of the challenges that faces designers of devices havinguser-engageable displays, such as touchscreen displays, pertains toproviding enhanced functionality for users, through gestures that can beemployed with the devices. This is so, not only with devices havinglarger or multiple screens, but also in the context of devices having asmaller footprint, such as tablet PCs, hand-held devices, mobile phone,smaller multi-screen devices and the like.

Due in part to the small size of some devices and touchscreens, thetypes and number “on-screen” gestures (e.g., gestures applied to atouchscreen) that can be provided by a particular device may be limited.Moreover, on-screen gestures may interfere with content presentations insome contexts, such as by occluding a video presentation or a portion ofdigital book a user is viewing. Alternative techniques such ascamera-based tracking and gestures may be impracticable or costprohibitive for some devices. Accordingly, traditional touch gesturesand input techniques may limit users and/or may be insufficient in somescenarios, use cases, or specific contexts of use.

SUMMARY

Photosensor array gesture detection techniques are described. In one ormore embodiments, a computing device includes an array of photosensors.The photosensor array can be configured in various ways to measurechanges in the amount of light that occur based upon a user's handposition above the photosensor array. In at least some embodiments,capacitance associated with the photosensors is charged and dataregarding discharge rates for the sensors is collected that isindicative of the amount of incident light. Sequential changes in theamount of light that is measured across the array of photosensors can beused to determine positioning and/or movement of the user's hand inthree dimensions (e.g., track position/motion in three-dimensional (3D)space relative to the computing device.) Accordingly, various gesturescan be defined in terms of input obtained via the photosensor array andrecognized to trigger corresponding operations by the computing device.

This Summary is provided to introduce a selection of concepts in asimplified form that are further described below in the DetailedDescription. This Summary is not intended to identify key features oressential features of the claimed subject matter, nor is it intended tobe used as an aid in determining the scope of the claimed subjectmatter.

BRIEF DESCRIPTION OF THE DRAWINGS

The detailed description is described with reference to the accompanyingfigures. In the figures, the left-most digit(s) of a reference numberidentifies the figure in which the reference number first appears. Theuse of the same reference numbers in different instances in thedescription and the figures may indicate similar or identical items.

FIG. 1 is an illustration of an example implementation of an environmentthat is operable to employ photosensor array gesture detectiontechniques described herein.

FIG. 2 depicts an example computing device that includes a photosensorarray.

FIG. 3 depicts an example implementation of a circuit for a photosensorarray of a gesture detection system.

FIG. 4 is a flow diagram depicting an example procedure to recognize agesture using a photosensor array in accordance with one or moreembodiments.

FIG. 5 is a flow diagram depicting an example procedure to define andrecognize gestures based on changes in photocurrent associated with aphotosensor array in accordance with one or more embodiments.

FIG. 6 is a flow diagram depicting an example procedure to implementcontent navigation gestures via a photosensor array that minimizeperceptible delay.

FIG. 7 illustrates various components of an example system that can beemployed in one or more embodiments to implement aspects of photosensorarray gesture detection techniques described herein.

DETAILED DESCRIPTION

Overview

Existing techniques for above surface gesture techniques may becomplicated and expensive. For instance, camera-based visual trackingtechniques may be impracticable for small devices, are relativelyexpensive, and/or may consume considerable processing, memory, and powerresources.

Photosensor array gesture detection techniques are described. In one ormore embodiments, a computing device includes an array of photosensors.The photosensor array can be configured in various ways to measurechanges in the amount of light that occur based upon a user's handposition above the photosensor array. In at least some embodiments,capacitance associated with the photosensors is charged and dataregarding discharge rates for the sensors is collected that isindicative of the amount of incident light. Sequential changes in theamount of light that is measured across the array of photosensors can beused to determine positioning and/or movement of the user's hand inthree dimensions (e.g., track position/motion in three-dimensional (3D)space relative to the computing device.) Accordingly, various gesturescan be defined in terms of input obtained via the photosensor array andrecognized to trigger corresponding operations by the computing device.

In the following discussion, an example environment is first describedthat is operable to employ the photosensor array gesture detectiontechniques described herein. Example procedures are then described,which may be employed in the example environment, as well as in otherenvironments. Accordingly, the example devices and procedures are notlimited to the example environment and the example environment mayincorporate devices and procedures in addition to the examples describedherein. Lastly, an example computing system is described that can beemployed to implement photosensor array gesture detection techniques inone or more embodiments.

Operating Environment

FIG. 1 is an illustration of an example operating environment 100 thatis operable to employ the techniques described herein. The operatingenvironment includes a computing device 102 having a processing system104 and computer-readable media 106 that is representative of variousdifferent types and combinations of media, memory, and storagecomponents and/or devices that may be associated with a computingdevice. The computing device 102 is further illustrated as including anoperating system 108 and one or more applications 110 that may reside onthe computer-readable media (as shown), may be implemented at leastpartially by one or more hardware elements, and/or may be executed viathe processing system 104. Computer-readable media 106 may include both“computer-readable storage media” and “communication media,” examples ofwhich can be found in the discussion of the example computing system ofFIG. 7. The computing device 102 may be configured as any suitablecomputing system and/or device that employ various processing systems104 examples of which are also discussed in relation to the examplecomputing system of FIG. 7.

In the depicted example, the computing device 102 includes a displaydevice 112 that may be configured as a touchscreen to enable touchscreenand gesture functionality based on positioning and motion of a user'shand 114. The applications 110 may include a display driver, a gesturemodule 116, and/or other modules operable to provide touchscreen andgesture functionality enabled by the display device 112. Accordingly,the computing device may be configured to recognize input and gesturesthat cause corresponding operations to be performed.

For example, a gesture module 116 may be configured to recognize a touchinput, such as a finger of a user's hand 114 as on or proximate to thedisplay device 112 of the computing device 102 using touchscreenfunctionality. The gesture module 116 also represents functionality forrecognition and handling of gestures based on input and data collectedvia a suitable photosensor array as described herein. Thus, a variety ofdifferent types of gestures may be recognized by the computing deviceincluding, by way of example and not limitation, gestures that arerecognized from a single type of input (e.g., touch gestures), gesturesinvolving multiple types of inputs, touch gestures applied to displaydevice 112, above surface gestures based on movement in athree-dimensional (3D) space that extends out from the display surface,and so forth. For example, the gesture module 116 can be utilized torecognize single-finger gestures and bezel gestures,multiple-finger/same-hand gestures and bezel gestures, and/ormultiple-finger/different-hand gestures and bezel gestures. Further, thecomputing device 102 may be configured to detect and differentiatebetween photosensor array input/gestures, touchscreen inputs, stylusinput, camera-based vision tracking gestures, and other different typesof inputs. Moreover, various kinds of inputs obtained from differentsources, including the photosensor array gestures, touch inputs, stylusinput, above surface gestures, and/or inputs obtained through a mouse,touchpad, software or hardware keyboard, and/or hardware keys of adevice (e.g., input devices), may be used in combination to causecorresponding device operations.

To implement photosensor array gesture detection techniques, thecomputing device 102 may further include a gesture detection system 118that includes or otherwise makes uses of a controller module 120 and aplurality of photosensors 122. The gesture detection system 118represent functionality operable to obtain and use various input fromthe photosensors 122 that is indicative of “above surface” gesturessupported by the gesture detection system 118. Generally speaking, thegesture detection system 118 employs an array of photosensors 122 tomeasure changes in light that occur as a user positions and moves theirhand above the display device 112. For instance, the controller module120 may selectively operate the photosensors 122 in different modes tocollect data regarding the amount of light incident upon thephotosensors 122. Changes in light levels may occur based on the amountof ambient light that is reflected or blocked when a user moves theirhand above the display and/or tilts the device. Changes in the amount ofincident light upon the photosensors 122 may be used to assesspositioning and/or movement of a user's hand(s) as wells as to sensetilt of the device. Various above surface gestures may be defined tocorrespond to particular patterns or sequences of light changes and handmovements relative to the array of sensors. Thus, detection ofparticular patterns or sequences of light changes can be correlated togestures and used to drive computing operations.

The controller module 120 may be implemented to provide logic to controloperation of the photosensors 122 and process input that is receivedthrough the photosensors 122. For example, the controller module 120 mayrepresent a microcontroller and/or other hardware used to control anarray of photosensors 122. Functionality of the controller module 120may alternatively be implemented at least partially as software that isexecuted by the processing system 104. The controller module 120 mayalso be further configured to supply input data that is collected fromthe photosensors 122 to the gesture module 116 to recognize variousgestures and cause corresponding actions. A suitable photosensor arraymay be implemented and arranged in various ways, examples of which arediscussed in relation to the following figures.

Above surface gestures as used herein refer to gestures in a threedimensional (3D) space extending out from the computing device and/ordisplay surface (e.g., gestures made at some distance above, in frontof, or otherwise away from the display surface). Such above surfacegestures can occur without touch-input, near-surface capacitive touch,and/or contact with the display device 112. Recognition of above surfacegestures through a photosensor array is therefore distinguishable fromrecognition of touchscreen input/gestures (e.g., “on-screen” gestures)applied to a display device 112 as discussed above. Moreover, using anarray of photosensors 122 provides a relatively inexpensive andstraightforward way to detect gestures above the surface of a device incomparison to existing techniques. For instance, existing camera-basedtechniques that rely upon a depth detection camera and visual objecttracking for above surface gesture detection are complex and consumeconsiderable processing and/or battery power. Techniques that rely uponexternal input devices, such as hand-held or wearable devices, mayprevent the user from freely using their hands and typically involvecommunication componentry and interactions between devices that can addcost and complicate processing. In contrast, the techniques to detectgestures described herein are not dependent upon manipulation by a userof or communication with external hand-held or wearable input devices.The photosensor array gesture detection techniques as described hereinenable natural hands-free gestures and can avoid complexity and expenseassociated with existing techniques.

Having described an example operating environment, consider now adiscussion of some example implementation details regarding aphotosensor array suitable to implement techniques for photosensor arraygesture detection in one or more embodiments.

Photosensor Array Details

In this section, details regarding a photosensor array are described inrelation to example illustrations of FIG. 2 and FIG. 3. A photosensorarray is generally configured to enable detection of gestures based uponuser movement above/away from the display surface of a computing device.Such above surfaces gestures may be used in addition to, in lieu of,and/or in combination with other kinds of input including touchscreeninput and input from various input devices.

In particular, FIG. 2 depicts generally at 200 an example computingdevice 102 of FIG. 1 that includes a photosensor array 202 having aplurality of photosensors 122. The plurality of photosensors 122depicted may represent individual sensors and/or clusters of multiplesensors. The photosensor array 202 may be arranged in various ways andat different locations. For instance, the photosensor array 202 may beprovided as an integrated component of the display device 112 (asdepicted), as an arrangement of sensors embedded into a housing for thecomputing device 102, as a separate external device or removable add-ondevice that can be connected to the computing device 102 via a wired orwireless interface, and so forth. In particular, example photosensors122 of the photosensor array 202 in FIG. 2 are depicted as beingarranged linearly across the computing device/display. Otherarrangements are also contemplated such as a uniform or non-uniform gridof sensors disposed throughout the display device 112, a staggeredarrangement, or other pattern for a plurality of photosensors 122.Although five photosensors 122 are depicted in FIG. 2, the number ofsensors for a photosensor array 202 may be different for variousdevices, displays, and/or arrangements. For example, the number ofsensors (or sensor clusters) may range from just a few sensors (e.g.,two to nine) to tens or even hundreds of sensors in differentembodiments.

A variety of suitable photosensors 122 may be employed. In one approach,the photosensors are configured as light emitting diodes (LEDS) thatoperate as light detectors. As may be appreciated by those of skill inthe art, LEDs may operate as light detectors by reverse biasing the LEDssuch that the voltage at the cathode of an LED circuit is higher than atthe anode of the LED circuit. This technique charges capacitance of theLEDs. Discharge rates for the capacitance of the LEDs can then bemonitored and analyzed to correlate the discharge rates to light levels,user hand position/motion, and/or corresponding gestures. Further, LEDsmay be cycled back and forth between light emitting and light detectionmodes at a rate that is imperceptible to the human eye. This enablesselected LEDs of the display device 112 to be used to implement thephotosensor array 202 as well as for display of content. Thus, existingLEDs or a display may be repurposed as photosensors and/or reconfiguredto act as both light emitters and light detectors. Additionally oralternatively, LEDs or other sensors of the device or display may beconfigured as dedicated light detectors that are not employed fordisplay of content. Adding or dedicating a relatively small number ofLEDs integrated with a display as light detectors generally does notcause significant or user detectable issues with display of content viathe display. Other kinds of photosensors 122 such as photodiodes,photovoltaic cells, photoresistors, and other photo sensitive lightdetection devices may be used in various embodiments. In at least someembodiments, the amount of photocurrent may be directly measured usingsuitably configured photosensors.

Gestures recognized via input from a photosensor array 202 may beemployed to control interaction with the computing device 102 indifferent ways. For instance, a variety of gestures may be defined tocontrol content that is presented via a user interface 204 on thedisplay device 112. The gestures may be defined in terms of light level(e.g., photocurrent) changes and sequences of changes that aredetectable via the photosensor array 202. By way of example and notlimitation, gestures may be used to control navigation of content,content/menu selections, views of the user interface 204, and so forth.This may include operations such as turning pages of a digital book,bookmarking content, navigating a photo library or other media library,playing games, zooming in/out, cut and paste operations, rearrangementsof icons or other content representations, menu item selections,navigation of operating system and application user interfaces,selecting/launching/closing and otherwise controlling execution ofapplications, and various other operations. In addition to detection ofgestures, the photosensor array 202 may also be employed as a generalmotion detector, as a tilt sensor for a device based on changes acrossmultiple sensors of the array, and/or to resolve positions of users(e.g., arms, hands, feet, fingers, etc.) as wells as objects (e.g.,stylus, pointer, wand, etc.) in 3D space extending out from the array.

One particular example input scenario using the photosensor array 202 isdepicted in FIG. 2. Here, a user's hand 114 is represented at 206 asmaking a waving motion generally from left to right across the upperright hand portion of the display. This waving motion may be defined asa gesture to manipulate the displayed user interface 204. In thisexample, the user interface 204 is configured as a picture viewer inwhich a vehicle picture is presented. In this scenario, the wavingmotion may cause navigation between various pictures in a folder orcollection. Thus, the vehicle picture may be changed to another picturein the collection based on the waving motion. In other examplescenarios, comparable gestures may correlate to navigation between openapplications in an operating system interface, navigation between pagesor tabs of a web browser, turning of pages of a digital book in areading application and so forth. The example waving motion and othermotions/positioning of the user's hand 114 cause the amount of lightincident upon the photosensors 122 to change accordingly. Thephotosensors 122 may capture these changes as changes in photocurrent.Generally, the photocurrent that is measured decreases as a user passestheir hand (or an object) over a photosensor and blocks ambient lightfrom reaching the photosensor. A controller module 120 may beimplemented by the gesture detection module 118 to analyze and interpretthe photocurrent changes for individual sensors and/or across differentsensors to recognize gestures and initiate corresponding actions.Additional details regarding these and other aspects of techniques forphotosensor array gesture detection may be found in relation to thefollowing figures.

FIG. 3 depicts generally at 300 an example implementation of a circuitfor a photosensor array 202 of a gesture detection system 118. In thisexample, the circuit includes a controller module 120 in a hardware formto implement detection logic 302 for collecting, analyzing and/orinterpreting input from the photo sensor array 202. The controllermodule 120 for example may be configured as microcontroller, such as aperipheral interface controller (PIC) microcontroller. Functionality ofthe controller module 120 and/or detection logic 302 may also beimplemented at least partially via a processing system 104 of a deviceand/or software executed by the processing system 104. The examplecircuit of FIG. 3 includes five photosensors configured as LEDs 304. TheLEDs 304 in the example circuit are each coupled to respectiveinput/output (I/O) pins on the microcontroller and to a resistor.Specifically the anode side of each LED 304 is coupled to a respectiveanode pin 306 of the controller module 120. The cathode side of each LED304 is coupled to a respective resistor 308 and a cathode pin 310.Although LEDs are depicted, other kinds of photosensors 122 may also beemployed. For instance, photoresistors may be employed for embodimentsin which the photosensors 122 are not configured for use as lightemitters/light sources.

Naturally, a circuit may include more or less photosensors in comparablearrangements. Additional photosensors may be used to improve sensitivityof the array and/or to enable increased ability to sense motion/positionin three-dimensions. For the purpose of this discussion athree-dimensional coordinate system may defined with x, y, and z axesrelative to the array and device surface where an x-y plane is definedby the display surface and the z-axis extends outward from the surfaceand may define height above the surface. For instance, in onearrangement thirty-six photosensors may be disposed in an array across adisplay device. A variety of other example arrangements that employdifferent numbers of photosensors are also contemplated. The number ofphotosensors employed for various applications of the describedtechniques may depend upon considerations including cost, device/displaysize, complexity of supported gestures, power consumption, position ofthe array, and so forth.

In operation, the detection logic 302 is configured to selectively applya charge (e.g., voltage) to the LEDs to charge capacitance of the LEDs,release the charge, and determine the amount of time it takes todischarge the LEDs. Longer discharge time corresponds to lessphotocurrent and less corresponding light being received at a particularphotosensor. More generally, the detection logic 302 may causemeasurement of the amount photocurrent through an array of photosensorsover time in various ways. This may be based on the rate of capacitivedischarge using LEDs, direct measurement of photocurrent withphotodiodes or photoresistors, and so forth.

To charge capacitance of the LEDs, the detection logic 302 may operateto alternate the pins of the microcontroller between states. Asmentioned, LEDs may be used as both light emitters and light detectors.In this approach, the detection logic 302 may cycle both the LED anodesand LED cathodes at a designated rate between output low and output highstates for each pin. In this approach, the anode pins 306 and cathodepins 310 are driven alternately to opposite states. In another approach,the anode pins 306 may be connected to ground and the cathode pins 310are cycled between high and low states.

In either of these cases, when the cathode side is in an output lowstate, the LEDs operate as light emitters. When the cathode side is inan output high state, the LEDs may operate as photosensors. Cyclingquickly back and forth between the low and high states enables the sameLEDS to alternate between emitting and sensing modes. A delay betweendifferent states enables collection of data regarding photocurrent byswitching pins to an input mode. A cycle of the states including thedelay may be configured to occur relatively quickly (e.g., withinmicroseconds) so the alternating is not visually perceptible by aviewer.

In particular, when the cathode pins 310 are in a high output state, theLEDs work as capacitors in parallel with a current source which modelsoptically induced photocurrent that can be measured as an indication ofabove surface gestures, motion and/or position. Cycling cathode pins 310from low to high charges the capacitance. Then, the cathode pins 310 maybe switched an input mode, which causes the photocurrent through thecircuit to discharge the capacitance of the LEDs. Timing the amount oftime is takes the capacitance of the LEDs to discharge down to athreshold level provides a measurement of the photocurrent andaccordingly the amount of incident light associated with each of theLEDs. Discharge times may be computed in any suitable way. For example,the detection logic 302 may start timers to measure the time it takesfor the LEDs to discharge. The times continue to run until thephotosensor is discharged to a threshold level. With less photocurrent,it takes longer to discharge and accordingly the measured amount of timewill be greater. Thus, timing data for discharge of each LED and eachone of multiple cycles may be collected. In some embodiments, the timingdata may be mapped to discrete light levels on a defined scale. Forinstance, a scale from one to ten may be defined to correlate dischargetimes and photocurrent measurements to defined light levels on thescale. The timing data and/or corresponding scale values may be storedas register values of the microcontroller associated with the LEDs, in adatabase, or otherwise.

Analysis of the collected timing data enables detection of gestures andother user interaction with a computing device 102. The timing datareflects the amount of light incident to particular photosensors. Thus,as a user positions and/or moves their hand over various photosensors,some ambient light may be blocked and time values may increaseaccordingly. Sequential changes and patterns in the timing data may beindicative of particular user action and may be correlated to definedgestures that trigger operations. Generally, patterns and sequentialchanges for individual sensors may be indicative of positioning and/ormotion up/down relative to the array (e.g., in a z direction extendingabove the display surface). Changes across multiple sensors may beindicative of gestures and motions across the display (e.g., in x or ydirections defined by the surface) For example, the hand waving gesturefrom left to right depicted in FIG. 2 may correlate to a sequentialdecrease in light level in order across the three sensors on the righthand side. Waving back and forth may be indicated by successive lightlevel changes in opposite directions across the three sensors.Positioning of a user's hand over a particular sensor may be indicatedby a decrease in light level that remains steady for a period of time.Subsequent changes in the light level for the particular sensor (orgroup of sensors) may indicate up/down movement of the users hand in thez direction, which could correlate to zooming out/in on an object orpage. Thus, various patterns and changes can be detected based oncollected timing data and used to drive corresponding operations.

Generally speaking, gestures are not instantaneous commands but aredefined by a series of corresponding states, which in the context ofthis document are different states for sensors of the photosensor array.The gestures may be defined in terms of a beginning state and end statefor the array and optionally one or more intermediate states. Waitingfor detection of the end state to trigger operation may potentiallyintroduce delay that may be visually perceived by a viewer. This mayoccur from example with animations associated with content navigation,such as picture viewing transitions and transitions for turning pages.

To handle this and minimize perceptible delay, operations for somegestures may be started in response to detection of a user's hand abovethe array at particular photosensors locations. The operations may thenbe completed when the full sequence of particular states matching agesture are detected. The particular gesture/operations that aretriggered may depend upon the interaction context. Consider, forexample, a page turning animation in the context of user interactionwith a digital book via a reader application. An initial sequence of thepage turning animation may be initiated as soon as the user's hand ispositioned and detected above an appropriate one of the sensors in thearray. For example, a designated number of frames for the animation maybe triggered based on detection of the initial position. This may givethe appearance of a page wiggling, beginning to turn, or otherwisegiving an indication that the page is ready for turning. The remainingframes of the animation may then be triggered upon the detection ofstates that match the page turning gesture. Here, the animationcontinues at a point after the designated number of frames so the pageturning proceeds from where it left off. This technique can be employedto minimize or eliminate perceptive delay for some gestures and/orcorresponding animations.

Having described some details regarding photosensor array gesturedetection techniques, consider now some example procedures in accordancewith one or more embodiments.

Example Procedures

The following discussion describes photosensor array gesture detectiontechniques that may be implemented utilizing the previously describedsystems and devices. Aspects of each of the procedures may beimplemented in hardware, firmware, software, or a combination thereof.The procedures are shown as a set of blocks that specify operationsperformed by one or more devices and are not necessarily limited to theorders shown for performing the operations by the respective blocks. Inportions of the following discussion, reference will be made to theenvironment 100 and examples of FIGS. 2 and 3, respectively. In at leastsome embodiments, the procedures may be performed by a suitablyconfigured computing device, such as the example computing device 102 ofFIG. 1 that includes or otherwise make use of a gesture detection system118.

FIG. 4 depicts an example procedure 400 to recognize a gesture using aphotosensor array in accordance with one or more embodiments. An arrayof photosensors associated with a computing device is initialized (block402). For example, a controller module 120 of a gesture detection system118 may operate to initialize a photosensor array 202 in various ways todetect above surface gestures for a computing device. This may involvecycling a circuit to charge capacitance of a various photosensors 122 inthe array as discussed previously. In addition or alternatively, voltagemay be applied to photosensors 122 configured to measure photocurrentdirectly to prepare the photosensors 122 to obtain information regardinglight incident to the photosensors 122. The photosensor array 202 may beprovided as an integrated component of the computing device 102 or as aseparate/removable component connectable to the computing device 102.

Data is collected regarding photocurrent for the array of photosensors(block 404). Here, the photosensor array 202 is used to collect dataregarding the amount of light at the surface of the computing device102. For instance, timing data regarding discharge rates for capacitanceof an LED array may be collected as discussed previously. The timingdata reflects photocurrent associated with particular sensors andtherefore corresponding light levels. Photocurrent information may alsobe collected directly or indirectly via photodiodes, photoresistors,and/or other kinds of photosensors. The collected data may be stored inany suitable manner in an appropriate location accessible to supply thecollected data for analysis.

A gesture is recognized based upon the collected data regarding thephotocurrent (block 406). Gestures may be recognized in any suitable waybased on data indicative of photocurrent that is collected. Inparticular, the controller module 120 may interact with a gesture module116 to interpret the collected data. This involves analyzing thecollected data to detect patterns and/or sequences of photocurrentchanges that correlate to particular gestures, some examples of whichwere discussed previously. Thus, a gesture supported by the gesturemodule 116 may be recognized based upon photocurrent input that isobtained via the photosensor array 202.

When a particular gesture is recognized, the recognition of the gesturecauses the computing device to perform operations corresponding to therecognized gesture (block 408). Various operations corresponding todifferent contexts, applications, user interfaces and content associatedwith a computing device 102 may be triggered. In general, gestures maybe used to control navigation of content, content/menu selections, viewsof the user interface, and so forth. In addition, tracking of changesacross an array of sensors may be correlated to a relative tilt positionof the computing device and motion of the device in 3D space (as opposedto motion of the user's hand). Thus, the collected data regardingchanges in photocurrent may also be used for tilt detection and/or toimplement motion detection for the computing device 102. Above surfacegesture may be particularly applicable in situation in which a user maybe unable to directly touch the display because a protective cover isbeing used, the user is positioned away from the device, and so forth.For instance, a waterproof case may prevent direct touches and abovesurface gestures may be used instead. Likewise, a user referencing acookbook for cooking or a manual to assemble a product or make repairscould make use of above surface gestures to avoid having to put downtools, place dirty hands on the screen, and/or take their hands away toofar away from the project at hand.

FIG. 5 depicts an example procedure 500 to define and recognize gesturesbased on changes in photocurrent associated with a photosensor array inaccordance with one or more embodiments. Gestures are defined thatcorrespond to changes in photocurrent for an array of photosensorscaused by movement of a user's hand over the array (block 502). Forexample, various gestures may be defined as described above. In at leastsome embodiments, a variety of gestures supported by a device may bedefined by a gesture module 116. Gestures may be defined in terms ofinput from a photosensor array as described herein as well as throughother techniques such as touch-based and/or camera-based input. Asmentioned, individual gestures may be defined based on a combination ofinput from multiple different input sources. In accordance withtechniques described herein, at least some gestures are defined in termsof sequences of changes in photocurrent (e.g., patterns) for one or morephotosensors in the array.

In particular, changes in photocurrent for the photosensor array areascertained (block 504). This may occur in any suitable way. In oneapproach, capacitance of the photosensors is selectively charged (block506) and the amount of time it takes to discharge the capacitance ofeach photosensor is measured (block 508). As described above, the timeassociated with discharge of capacitance correlates to the amount oflight incident to a sensor and accordingly to photocurrent. Less lighttranslates to less photocurrent and a longer discharge time. In someembodiments, values for photocurrent or light levels may be obtaineddirectly from suitably configured sensors. In any case, data iscollected that is indicative of changes in photocurrent for thephotosensor array over time. Changes and patterns over time inphotocurrent can be derived from timing data or directly fromphotocurrent data when available.

A defined gesture is recognized that corresponds to the ascertainedchanges (block 510). For example, the gesture detection system 118 maythen interact with the gesture module 116 to match detectedchanges/patterns to corresponding gestures and/or to coordinate withinput obtained from different input sources. The recognized gesture maytrigger corresponding computing operations in a variety of contexts,examples of which were previously described.

FIG. 6 depicts an example procedure 600 to minimize perceptible delayfor content navigation gestures detected via a photosensor array.Positioning of a user's hand relative to a photosensor in an array ofphotosensors is detected (block 602). For instance, relatively steadyhovering of a user's hand above a particular sensor may be detected inthe manner previously described. In a content navigation context, thisaction may be indicative of the user preparing to navigate, such as toturn a page of digital book, swipe through a media collection, navigatea system menu, interact with browser pages, and so forth. An initialcontent navigation animation associated with a gesture is triggered inresponse to detection of the positioning (block 604). Here, a bifurcatedapproach may be taken to minimize or eliminate perceptible delay asdiscussed previously. This involves performing operations correspondingto a gesture in stages. Rather than waiting until an end state for agesture is detected, a portion of the operations corresponding to agesture may be triggered as soon as the user positions their hand forthe gesture. In the case of navigation animations, this may involverendering a selected number of the initial frames for the animation, forexample the first ten or so frames. This prepares the system to completethe animation when appropriate input is detected and may provide visualindications of the availability of the gesture to the user. In othercontexts, preparation to complete operations may be initiated in acomparable manner. For example, in an object selection context a bordermay initially be placed around an object in response to positiondetection. Likewise, a picture transition animation such as a fadeeffect or fly-out effect may be initiated based on position detection ina picture viewing context. A variety of other examples are alsocontemplated.

The gesture is detected based upon sequential photocurrent changes forone or more photosensors of the array (block 606) and content isnavigated according to the detected gesture (block 610). Here, after theinitial position detection, the full sequence of photocurrent changesthat defines the gesture may be detected in the manner describedpreviously. In response, remaining operations for the gesture arecompleted to manipulate content in accordance with the particulargesture. The operations for the gesture may be continued at a pointfollowing the operations performed in the initial stage. Thus, in thepreceding content navigation animation example, the animation maypick-up with the eleventh frame to complete the operations. In this way,perceptible delay associated with some gestures may be reduced oreliminated.

Having discussed some example procedures, consider now an example systemthat can be employed in one or more embodiments to implement aspects ofphotosensor array gesture detection techniques described herein.

Example System

FIG. 7 illustrates an example system 700 that includes an examplecomputing device 702 that is representative of one or more computingsystems and/or devices that may implement the various techniquesdescribed herein. The computing device 702 may be, for example, a serverof a service provider, a device associated with a client (e.g., a clientdevice), an on-chip system, and/or any other suitable computing deviceor computing system.

The example computing device 702 as illustrated includes a processingsystem 704, one or more computer-readable media 706, and one or more I/Ointerfaces 708 that are communicatively coupled, one to another.Although not shown, the computing device 702 may further include asystem bus or other data and command transfer system that couples thevarious components, one to another. A system bus can include any one orcombination of different bus structures, such as a memory bus or memorycontroller, a peripheral bus, a universal serial bus, and/or a processoror local bus that utilizes any of a variety of bus architectures. Avariety of other examples are also contemplated, such as control anddata lines.

The processing system 704 is representative of functionality to performone or more operations using hardware. Accordingly, the processingsystem 704 is illustrated as including hardware elements 710 that may beconfigured as processors, functional blocks, and so forth. This mayinclude implementation in hardware as an application specific integratedcircuit or other logic device formed using one or more semiconductors.The hardware elements 710 are not limited by the materials from whichthey are formed or the processing mechanisms employed therein. Forexample, processors may be comprised of semiconductor(s) and/ortransistors (e.g., electronic integrated circuits (ICs)). In such acontext, processor-executable instructions may beelectronically-executable instructions.

The computer-readable media 706 is illustrated as includingmemory/storage 712. The memory/storage 712 represents memory/storagecapacity associated with one or more computer-readable media. Thememory/storage 712 may include volatile media (such as random accessmemory (RAM)) and/or nonvolatile media (such as read only memory (ROM),Flash memory, optical disks, magnetic disks, and so forth). Thememory/storage 712 may include fixed media (e.g., RAM, ROM, a fixed harddrive, and so on) as well as removable media (e.g., Flash memory, aremovable hard drive, an optical disc, and so forth). Thecomputer-readable media 706 may be configured in a variety of other waysas further described below.

Input/output interface(s) 708 are representative of functionality toallow a user to enter commands and information to computing device 702,and also allow information to be presented to the user and/or othercomponents or devices using various input/output devices. Examples ofinput devices include a keyboard, a cursor control device (e.g., amouse), a microphone for voice operations, a scanner, touchfunctionality (e.g., capacitive or other sensors that are configured todetect physical touch), a camera (e.g., which may employ visible ornon-visible wavelengths such as infrared frequencies to detect movementthat does not involve touch as gestures), and so forth. Examples ofoutput devices include a display device (e.g., a monitor or projector),speakers, a printer, a network card, tactile-response device, and soforth. Thus, the computing device 702 may be configured in a variety ofways as further described below to support user interaction.

Various techniques may be described herein in the general context ofsoftware, hardware elements, or program modules. Generally, such modulesinclude routines, programs, objects, elements, components, datastructures, and so forth that perform particular tasks or implementparticular abstract data types. The terms “module,” “functionality,” and“component” as used herein generally represent software, firmware,hardware, or a combination thereof. The features of the techniquesdescribed herein are platform-independent, meaning that the techniquesmay be implemented on a variety of commercial computing platforms havinga variety of processors.

An implementation of the described modules and techniques may be storedon or transmitted across some form of computer-readable media. Thecomputer-readable media may include a variety of media that may beaccessed by the computing device 702. By way of example, and notlimitation, computer-readable media may include “computer-readablestorage media” and “communication media.”

“Computer-readable storage media” may refer to media and/or devices thatenable persistent and/or non-transitory storage of information incontrast to mere signal transmission, carrier waves, or signals per se.Thus, computer-readable storage media refers to non-signal bearingmedia. The computer-readable storage media includes hardware such asvolatile and non-volatile, removable and non-removable media and/orstorage devices implemented in a method or technology suitable forstorage of information such as computer readable instructions, datastructures, program modules, logic elements/circuits, or other data.Examples of computer-readable storage media may include, but are notlimited to, RAM, ROM, EEPROM, flash memory or other memory technology,CD-ROM, digital versatile disks (DVD) or other optical storage, harddisks, magnetic cassettes, magnetic tape, magnetic disk storage or othermagnetic storage devices, or other storage device, tangible media, orarticle of manufacture suitable to store the desired information andwhich may be accessed by a computer.

“Communication media” may refer to a signal-bearing medium that isconfigured to transmit instructions to the hardware of the computingdevice 702, such as via a network. Communication media typically mayembody computer readable instructions, data structures, program modules,or other data in a modulated data signal, such as carrier waves, datasignals, or other transport mechanism. Communication media also includeany information delivery media. The term “modulated data signal” means asignal that has one or more of its characteristics set or changed insuch a manner as to encode information in the signal. By way of example,and not limitation, communication media include wired media such as awired network or direct-wired connection, and wireless media such asacoustic, RF, infrared, and other wireless media.

As previously described, hardware elements 710 and computer-readablemedia 706 are representative of instructions, modules, programmabledevice logic and/or fixed device logic implemented in a hardware formthat may be employed in some embodiments to implement at least someaspects of the techniques described herein. Hardware elements mayinclude components of an integrated circuit or on-chip system, anapplication-specific integrated circuit (ASIC), a field-programmablegate array (FPGA), a complex programmable logic device (CPLD), and otherimplementations in silicon or other hardware devices. In this context, ahardware element may operate as a processing device that performsprogram tasks defined by instructions, modules, and/or logic embodied bythe hardware element as well as a hardware device utilized to storeinstructions for execution, e.g., the computer-readable storage mediadescribed previously.

Combinations of the foregoing may also be employed to implement varioustechniques and modules described herein. Accordingly, software,hardware, or program modules including the operating system 108,applications 110, gesture module 116, gesture detection system 118,controller module 120, and other program modules may be implemented asone or more instructions and/or logic embodied on some form ofcomputer-readable storage media and/or by one or more hardware elements710. The computing device 702 may be configured to implement particularinstructions and/or functions corresponding to the software and/orhardware modules. Accordingly, implementation of modules as a modulethat is executable by the computing device 702 as software may beachieved at least partially in hardware, e.g., through use ofcomputer-readable storage media and/or hardware elements 710 of theprocessing system. The instructions and/or functions may beexecutable/operable by one or more articles of manufacture (for example,one or more computing devices 702 and/or processing systems 704) toimplement techniques, modules, and examples described herein.

As further illustrated in FIG. 7, the example system 700 enablesubiquitous environments for a seamless user experience when runningapplications on a personal computer (PC), a television device, and/or amobile device. Services and applications run substantially similar inall three environments for a common user experience when transitioningfrom one device to the next while utilizing an application, playing avideo game, watching a video, and so on.

In the example system 700, multiple devices are interconnected through acentral computing device. The central computing device may be local tothe multiple devices or may be located remotely from the multipledevices. In one embodiment, the central computing device may be a cloudof one or more server computers that are connected to the multipledevices through a network, the Internet, or other data communicationlink.

In one embodiment, this interconnection architecture enablesfunctionality to be delivered across multiple devices to provide acommon and seamless experience to a user of the multiple devices. Eachof the multiple devices may have different physical requirements andcapabilities, and the central computing device uses a platform to enablethe delivery of an experience to the device that is both tailored to thedevice and yet common to all devices. In one embodiment, a class oftarget devices is created and experiences are tailored to the genericclass of devices. A class of devices may be defined by physicalfeatures, types of usage, or other common characteristics of thedevices.

In various implementations, the computing device 702 may assume avariety of different configurations, such as for computer 714, mobile716, and television 718 uses. Each of these configurations includesdevices that may have generally different constructs and capabilities,and thus the computing device 702 may be configured according to one ormore of the different device classes. For instance, the computing device702 may be implemented as the computer 714 class of a device thatincludes a personal computer, desktop computer, a multi-screen computer,laptop computer, netbook, and so on.

The computing device 702 may also be implemented as the mobile 716 classof device that includes mobile devices, such as a mobile phone, portablemusic player, portable gaming device, a tablet computer, a multi-screencomputer, and so on. The computing device 702 may also be implemented asthe television 718 class of device that includes devices having orconnected to generally larger screens in casual viewing environments.These devices include televisions, set-top boxes, gaming consoles, andso on.

The techniques described herein may be supported by these variousconfigurations of the computing device 702 and are not limited to thespecific examples of the techniques described herein. This isillustrated through inclusion of the gesture detection system 118 on thecomputing device 702. The functionality represented by the gesturedetection system 118 and other modules may also be implemented all or inpart through use of a distributed system, such as over a “cloud” 720 viaa platform 722 as described below.

The cloud 720 includes and/or is representative of a platform 722 forresources 724. The platform 722 abstracts underlying functionality ofhardware (e.g., servers) and software resources of the cloud 720. Theresources 724 may include applications and/or data that can be utilizedwhile computer processing is executed on servers that are remote fromthe computing device 702. Resources 724 can also include servicesprovided over the Internet and/or through a subscriber network, such asa cellular or Wi-Fi network.

The platform 722 may abstract resources and functions to connect thecomputing device 702 with other computing devices. The platform 722 mayalso serve to abstract scaling of resources to provide a correspondinglevel of scale to encountered demand for the resources 724 that areimplemented via the platform 722. Accordingly, in an interconnecteddevice embodiment, implementation of functionality described herein maybe distributed throughout the system 700. For example, the functionalitymay be implemented in part on the computing device 702 as well as viathe platform 722 that abstracts the functionality of the cloud 720.

CONCLUSION

Although the invention has been described in language specific tostructural features and/or methodological acts, it is to be understoodthat the invention defined in the appended claims is not necessarilylimited to the specific features or acts described. Rather, the specificfeatures and acts are disclosed as example forms of implementing theclaimed invention.

What is claimed is:
 1. A method comprising: initializing an array ofphotosensors associated with a computing device; collecting dataregarding photocurrent for the array of photosensors; recognizing agesture based upon the collected data regarding the photocurrent, thegesture defined in terms of photocurrent changes responsive to changesin the amount of ambient light incident upon the array of photosensorsover time, the changes in the amount of ambient light including changescaused by blocking the ambient light; causing the computing device toperform operations corresponding to the recognized gesture.
 2. A methodas described in claim 1, wherein the array of photosensors compriseslight emitting diodes (LEDs) configured to operate as both lightemitters and light detectors.
 3. A method as described in claim 1,wherein the array of photosensors comprises light emitting diodes (LEDs)configured to operate as dedicated light detectors.
 4. A method asdescribed in claim 1, wherein the array of photosensors comprisesphotoresistors.
 5. A method as described in claim 1, wherein the arrayof photosensors is integrated with a display device of the computingdevice.
 6. A method as described in claim 1, wherein the array ofphotosensors is configured as a separate device that is connectable tothe computing device.
 7. A method as described in claim 1, wherein theinitializing comprises charging capacitance associated with thephotosensors.
 8. A method as described in claim 7, wherein collectingdata regarding photocurrent comprises collecting data regarding changesin photocurrent for the photosensors based upon discharge rates for thecapacitance associated with the photosensors.
 9. A method as describedin claim 1, wherein collecting data regarding photocurrent comprisesmeasuring sequential changes in light levels for individual sensorsacross the array of photosensors.
 10. A computing device comprising: adisplay device; a photosensor array having a plurality of photosensors;and a gesture detection system configured to: define gestures thatcorrespond to changes in photocurrent produced at least in part byblocking the ambient light reaching the plurality of photosensors in thephotosensor array, the blocking being caused by movement of a user'shand over the photosensor array; and ascertain changes in thephotocurrent for the photosensor array to recognize the defined gesturesand cause the computing device to perform corresponding operations. 11.A computing device as described in claim 10, wherein the plurality ofphotosensors comprises light emitting diodes (LEDs).
 12. A computingdevice as described in claim 10, wherein the plurality of photosensorsis embedded in a housing for the computing device.
 13. A computingdevice as described in claim 10, wherein the plurality of photosensorsis arranged as an integrated component of the display device.
 14. Acomputing device as described in claim 10, wherein the gesture detectionsystem further includes a controller module to ascertain the changes inthe photocurrent by: selectively charging capacitance of the pluralityof photosensors; and measuring an amount of time it takes eachphotosensor to discharge.
 15. A computing device as described in claim10, wherein the gesture detection system further comprises amicrocontroller having device logic to control operation of thephotosensor array and input/output pins to which each of the pluralityof photosensors is connected.
 16. A computing device as described inclaim 15, wherein the microcontroller is configured to ascertain thechanges by: cycling input/output pins on a cathode side of the pluralityof photosensors from a low output state to a high output state to chargecapacitance of the plurality of photosensors; switching the input/outputpins on the cathode side to an input state to discharge to capacitance;and starting timers to measure an amount of time it takes to dischargethe capacitance of each of the plurality of photosensors.
 17. Acomputing device as described in claim 10, wherein the changes in thephotocurrent for the array are indicative of positioning and motion of auser's hand in three dimensional space relative to a surface of thedisplay device.
 18. One or more computer-readable storage media storinginstructions that, when executed by a computing device, cause thecomputing device to implement a gesture detection system configured toperform operations including: collecting photocurrent data via an arrayof light emitting diodes (LEDs) configured to operate as photosensors,the photocurrent data indicative of changes in light levels for thearray of LEDs over time that correlate to interaction of a user with thecomputing device, the changes in light levels being caused, at least inpart, by blocking the light; analyzing the collected photocurrent datato detect patterns of changes in the light levels for the array of LEDsindicative of particular user interaction with the computing device;recognizing an above surface gesture corresponding to a pattern ofchanges in the light levels that is detected based on the analyzing; andcausing the computing device to perform operations corresponding to therecognized above surface gesture.
 19. One or more computer-readablestorage media as described in claim 18, wherein the gesture detectionsystem is further configured to perform operations including detectingtilt of the computing device in three-dimensional space that correspondsto a pattern of changes in the light levels that is detected based onthe analyzing.
 20. One or more computer-readable storage media asdescribed in claim 18, wherein collecting photocurrent data via thearray of light emitting diodes (LEDs) comprises: cycling a cathode sideof a circuit for the LEDs between low and high output states to chargecapacitance associated with the LEDs in multiple cycles; and during eachof the multiple cycles: switching the cathode side of the circuit to aninput state to discharge the capacitance of the LEDs; and timing thedischarge of the capacitance of the LEDs to measure the amount ofphotocurrent associated with each of the LEDs.